Combining Machine Learning and Crowdsourcing for Better Understanding Commodity Reviews
نویسندگان
چکیده
In e-commerce systems, customer reviews are important information for understanding market feedbacks on certain commodities. However, accurate analyzing reviews is challenging due to the complexity of natural language processing and informal descriptions in reviews. Existing methods mainly focus on studying efficient algorithms that cannot guarantee the accuracy for review analysis. Crowdsourcing can improve the accuracy of review analysis while it is subject to extra costs and low response time. In this work, we combine machine learning and crowdsourcing together for better understanding customer reviews. First, we collectively use multiple machine learning algorithms to pre-process review classification. Second, we select the reviews on which all machine learning algorithms cannot agree and assign them to humans to process. Third, the results from machine learning and crowdsourcing are aggregated to be the final analysis results. Finally, we perform real experiments with practical review data to confirm the effectiveness of our method.
منابع مشابه
Making Better Use of the Crowd
This tutorial provides a comprehensive overview of the landscape of crowdsourcing research, targeted at the machine learning community. We begin with a showcase of innovative uses of crowdsourcing, covering direct applications to machine learning systems, crowdsourcing for hybrid intelligence, and large scale studies of human behavior online. We then dig into recent research aimed at understand...
متن کاملReducing Label Cost by Combining Feature Labels and Crowdsourcing
Decreasing technology costs, increasing computational power and ubiquitous network connectivity are contributing to an unprecedented increase in the amount of publicly available data. Yet this surge of data has not been accompanied by a complementary increase in annotation. This lack of annotated data complicates data mining tasks in which supervised learning is preferred or required. In respon...
متن کاملMaking Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research
This survey provides a comprehensive overview of the landscape of crowdsourcing research, targeted at the machine learning community. We begin with an overview of the ways in which crowdsourcing can be used to advance machine learning research, focusing on four application areas: 1) data generation, 2) evaluation and debugging of models, 3) hybrid intelligence systems that leverage the compleme...
متن کاملCombining human and machine intelligence in large-scale crowdsourcing
We show how machine learning and inference can be harnessed to leverage the complementary strengths of humans and computational agents to solve crowdsourcing tasks. We construct a set of Bayesian predictive models from data and describe how the models operate within an overall crowdsourcing architecture that combines the efforts of people and machine vision on the task of classifying celestial ...
متن کاملCrowdsourcing for ICD10 Code to Concept Relationships
In this work we leverage crowdsourcing in connection with machine learning techniques to validate candidate ICD10 Code to UMLS concept relationships that we generate. Our immediate use is in natural language understanding and machine learning approaches to automatically code electronic health record documents with ICD codes. Beyond auto-coding, the relationships will aid a wide variety of futur...
متن کامل